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Abstract. As an international informatics contest, or challenge, Bebras has started the second 
decade of its existence. The contest attracts more and more countries every year, recently there 
have been over 40 participating countries. From a single contest-focused annual event Bebras 
developed to a multifunctional challenge and an activities-based educational community build¬ 
ing model. This paper aims to introduce the Bebras model using ten years of observations in 
implementing the contest in different countries. The model is essentially based on democratic 
and inclusive education values. Systematic literature review of research papers concerning Be¬ 
bras activities has made an integral background for this model. The model is represented both 
at international and national levels and consists of several components where the development 
of Bebras tasks has taken a very significant role. Reasoning on innovated learning informatics 
and strengthening computational thinking by utilising carefully selected informatics concepts is 
discussed as well. 

Keywords: Bebras challenge, computational thinking, contest, democratic education, informatics 
concepts, informatics education, systematic literature review. 


1. Introduction 

Attracting youngsters to choose to study computer science or informatics - an equiva¬ 
lent terminology often used within Europe - at schools has always been a challenge 
for educators. The idea of developing contest on informatics fundamentals for school 
students was raised by Lithuania in 2004 (Dagiene, 2005; 2006). Recently, the contest 
has been spreading over 40 countries involving various activities, e.g. several rounds 
of the contest, discussion on informatics topics, task solving seminars, teacher work¬ 
shops, and task developing events. Since 2015, Bebras contest on informatics and 
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computer fluency was renamed to Bebras challenge on informatics and computational 
thinking (bebras.org). 

The contest name “bebras” is a Lithuanian word for “beaver” and it was chosen in 
connection with the hard-working, intelligent, goal seeking and lively animal. Some 
countries use the “bebras” for their national contest (for example, Azerbaijan, Iran, Ire¬ 
land, Italy, Japan, Spain, Taiwan, USA), and some countries translated it to their lan¬ 
guages (Estonia - kobras, Finland - majava, France - castor, Germany - biber, Serbia 
- dabar, Slovakia - bobr, UK -beaver, etc.). 

The main goal of the Bebras challenge is to motivate pupils to be interested in in¬ 
formatics topics and to promote thinking which is algorithmic, logical, operational, and 
based on informatics fundamentals. The Bebras activities have to promote pupils’ in¬ 
terest in informatics (also in information technology) at the early stage of the school 
education and to motivate pupils to learn deeper and master better technology (Dagiene, 
Futschek, 2008). Specifically, the idea is to encourage children to learn informatics fun¬ 
damentals (concepts), and to support development of algorithmic thinking as well as 
computational thinking. 

Our aspiration is to wrap up serious scientific problems of informatics and the basic 
concepts into playful tasks, inventive questions thus attracting students’ attention. It is 
not an easy matter to prepare tasks in such a way. Representatives of the Bebras coun¬ 
tries create and offer the assignments whole year long, and in spring they come together 
to discuss and choose the most suitable ones for the contest in autumn. 

Development of the Bebras tasks assignments is even more difficult concerning 
the fact that teaching of informatics is very different at schools in different countries. 
Naturally informatics is too young discipline to have adopted common agreements and 
clearly crystallised topics in different countries. We, the scientists of informatics, have 
to discuss, consult and find common practices and conventions. 

Thus, one of the most serious desires of Bebras is to find a place for informatics at 
schools, to reach an agreement on the principal topics and notions. Therefore, we are 
pleased to have succeeded in uniting a team of enthusiastic education scientists and ex¬ 
perts who are interested in informatics science. 

The Bebras challenge has been developed for all primary and secondary school 
pupils. The contest is meant to be performed in schools at computers. Pupils should 
solve 18 to 24 tasks within 45 to 55 minutes. There are different task sets for the age 
groups: 

• Little Beavers (grades 3-4). 

• Benjamin (grades 5-6). 

• Cadet (grades 7-8). 

• Junior (grades 9-10). 

• Senior (grades 11-12). 

The participants are usually supervised by teachers who may integrate the contest in 
their teaching activities. 

In the past few years, the number of the Bebras participants has been notably grow¬ 
ing. It outreach ed 1.3 million of pupils during contest in autumn 2015. 
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2. Bebras - an Inclusive Education Model for Learning Informatics Together 

The Bebras challenge is an education community network which consolidates over 40 
countries to discuss together informatics concepts for school informatics education. The 
Bebras task developing workshops organised annually (since 2005) bring together the 
representatives of all these countries for hard work and making decisions on good tasks 
for promoting informatics education at primary and secondary schools. 

From one side, the Bebras challenge is an international assembly driven by informat¬ 
ics education needs over the world. From the other side, almost all activities are based 
nationally, organised by country communities. Working together more than a decade, we 
have discovered principles of a democratic and inclusive education and a huge potential 
of creativity within the Bebras community. 

As many education theorists have agreed, a democratic education is an educational 
ideal in which democracy is both a goal and a method of instruction. It brings democratic 
values to education and can include self-determination within a community of equals, as 
well as such values as justice, respect and trust. Democratic education sees young peo¬ 
ple not as passive recipients of knowledge, but rather as active co-creators of their own 
learning. They are not the products of an education system, but rather valued participants 
in a vibrant learning community (Caine, 2009). 

The main ideas of the democratic education are discussed in a Waghit’s book (2014). 
Drawing on the works of poststructuralist theorists, Waghit argued that a democratic 
education in becoming has the potential to rupture pedagogical encounters towards new 
beginnings on the basis that teachers and students can never know with certainty and 
completeness. In democratic education, students and teachers are provoked to contribute 
to the community building and enhance their young citizens 21 st century skills. 

Consequently, many educators argued that teaching and learning in the democratic 
education ought to be associated with pedagogical activities in the making and solving 
everyday life tasks. We would like to add more concerning tasks: the tasks should be 
interesting for children, related to their activities, containing science (e.g. informatics) 
concepts, fun, etc. Additionally, we would like to extend democratic education to a 
mixed model of informal societies (e.g. the Bebras community) or groups of people with 
common interests to learn and gain some specific competencies. 

Observing ten years’ developments of the Bebras contests and activities, we elabora¬ 
ted a Bebras challenge model based on democratic education ideas and constructionism 
principles. Constructionism, as a learning approach (Ignatova, Dagiene, Kubilinskiene, 
2015), is mostly applied to the development of tasks and task solving activities for stu¬ 
dents as well as teachers. The model combines both international and national levels and 
involves variety of activities especially at country level (Fig. 1). 

Management of the Bebras challenge consists of three main bodies at national and 
international levels: National Bebras Organisation (NBO), International Bebras Com¬ 
munity (1BC) and the Bebras Board (BB). 

The National Bebras Organisation is responsible for many activities within a country 
and all year round: creating and submitting new tasks, reviewing tasks, participating in 
Bebras task evaluation workshops, selecting and translating tasks, organising challenge, 
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producing teaching material and Bebras task brochures, training teachers. There may be 
only one NBO per country. 

The International Bebras Committee is composed of NBO representatives. During 
the annual meeting, the international Bebras workshop, all NBO delegates are consid¬ 
ered 1BC members. For all IBC actions to be taken in between the workshops, each NBO 
nominates one representative as a voting member of the IBC. 

The Bebras Board is the executive body of the Bebras community and is responsible 
for all ongoing business of the community. The main functions are to take care of tasks 
(to develop new categories or patterns, new tools, etc.), especially to support the Bebras 
task evaluation workshop, and provide feedback to the community. 

The members of the Bebras community are organisations responsible for organising 
a national Bebras challenge in their country. 

The model has time constraints: some activities are strongly restricted by time limits 
(in months). For example, an annual task evaluation workshop should be organised at 
the end of May, so two months before, the preparation of high quality tasks should start. 
Following that, all new tasks should be delivered until the end of March - task review¬ 
ing process is taking place then. Each participating country should provide from 5 to 10 
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tasks. Anybody of the Bebras community may act as a reviewer. At least 2 reviewers are 
assigned for each proposed task. 

After tasks reviewing process finishes, discussions on accepted and recommended 
tasks begins. Usually it takes place during the first and second days of the international 
task evaluation workshop. 

The NBO must select and translate tasks for the annual Bebras contest. The second 
full week of November is considered as an official time for the Bebras challenge (first 
round) in participating country and named by International Bebras Week (for countries 
in the southern hemisphere, a different week may be chosen) (Bebras Community Sta¬ 
tutes, 2015). 

Several countries have established a second round of the Bebras challenge, usually at 
the end of January or beginning of February, and it is dedicated for the best participants 
of the first round challenge. Usually, the first and second rounds are followed by diffe¬ 
rent coming together meetings and awards for the best students. 

Teacher training is one of very important activities of local Bebras communities. Coun¬ 
tries are continually working on developing modern approaches for informatics teacher 
training, producing learning materials, organising hands-on seminars, etc. (Schulte et ah, 
2012). Thousands of students are involved in solving the Bebras tasks. It is a good possi¬ 
bility to collect enormous amounts of data on student performance and investigate them. 

Tasks are the most important and exceptional part of the Bebras model. The quality of 
tasks is crucial for the success of a contest. Usually, contests have several goals and the 
tasks have to fulfil a wide variety of criteria (Hakulinen, 2011). The tasks must reflect the 
goals of the competition and should be adequate for the applicants. Seeking to motivate 
students to learn science issues more deeply, contests are one of the best ways to capture 
their attention (Dagiene, 2008). In educational contests, tasks should attract students and 
drive them to learn and explore as well to develop skills in the particular area. 

Bebras tasks are short, answerable in a few minutes through a computerised inter¬ 
face, and requiring deep-thinking skills in the informatics field. The tasks should be 
answered without prior knowledge in informatics, and they are clearly related to fun¬ 
damental informatics concepts. To solve those tasks, pupils are required to think in and 
about information, discrete structures, computation, data processing, data visualisation, 
and they should use algorithmic as well as programming concepts. Each Bebras task 
can both demonstrate an aspect of informatics and test the participant’s ability of under¬ 
standing informatics fundamentals (Dagiene et al., 2015). 

Each year the international Bebras task evaluation workshop confirms a set of new 
tasks and questions. International experts in informatics follow a process that allows cre¬ 
ativity in finding new tasks and ensures a high quality of the output. It is not easy to cre¬ 
ate tasks that fulfil all criteria. Often it is a process of several versions from an imperfect 
task formulation to an acceptable formulation. The main criteria for tasks are: the task 
can be solved within 3 minutes; the problem statement is easy understandable; the task is 
presentable at a single screen page; the task is independent from specific systems. 

The easy understandability of tasks is in all contexts a very important goal. It is not 
only the wording but also the presentation of the task that may include interactive ele¬ 
ments. Since the pupils should be able to solve a task in an average time of 3 minutes, the 
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formulation of a task should be as short as possible but at any rate no longer than a single 
screen page. Tasks should also be funny and should have interactive elements. 

The interactivity is very typically for computers, so it is clear that a computer ori¬ 
ented contest should apply interactive elements to explain or solve tasks. Very often 
these interactive elements are “funny” to use and make the understandability of the 
problem statement much easier. However, the interactive elements are time consum¬ 
ing especially while implementing tasks in different systems. Anyway, due to the at¬ 
tractiveness of interactive tasks, the high effort for implementing the interactive part is 
worth to be done. 


3. Literature Analysis 

Inquiring data and improving technology for running the contest or developing tasks are 
questions that many countries are started to look at. Many research works are done. For 
better understanding in which direction the research concerning the Bebras challenge 
is going, we decided to make revision of research publications by using a systematic 
literature review (EPPI-Centre, 2007). 

Sources : Web of Science, ACM Digital Library, Springer Link, SCOPUS, Google 
Scholar. Search term : Beaver contest OR Bebras contest OR Bebras challenge. Lan¬ 
guage : English. There was no time limitation for publication. The Bebras was establish 
in 2004, and the first publication appeared in 2005 (Fig. 2). 

149 papers on informatics Bebras were found, but only 76 met the language criteria. 
The remaining 73 papers were published in different languages (Czech, Finnish, French, 
German, Italian, Japanese, Lithuanian, Russian, Slovakian, Slovenian, and Mandarin 
Chinese). 

In this analysis, the selected 76 papers are used. The papers are divided into three 
categories: 



Fig. 2. Number of papers by years and categories. 
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1. Bebras contest is the main source for research questions (39 papers). 

2. Bebras contest is discussed as a good practice for informatics education (17 pa¬ 
pers). 

3. Bebras contest is only mentioned between the other activities (20 papers). 
Number of research papers is significantly growing during the last years (Fig. 2). 

At first, all the 76 papers were analysed by their keywords in order to find out the 

key topics of the Bebras challenge. Naturally informatics education, computer science, 
programming, contest, learning, computational thinking and problem solving are the 
most dominating topics (Fig. 3). 

Further we are going to analyse 56 papers where the Bebras challenge is the main 
source for research questions or reasoning of good informatics education practices. 

The review was driven by the following research questions: 

1. What evidence in the literature is there to identify that the Bebras contest is ef¬ 
fective enough in promoting informatics concepts and computational thinking in 
children and young people? 

2. What evidence in the literature is there how to develop the quality of the Bebras 


contest? 


3. What evidence in the literature is there to see the impact of the Bebras contest for 
formal and/or informal informatics education in countries? 

One of the first papers was aimed to introduce the Bebras contest idea (Dagiene, 
2005). The main message for readers was to propose that competitions can play an im¬ 
portant role as a source of inspiration and innovation. A contest may be a key to create 
new knowledge and become an attractive way to bind up technology and education. 
There was an invitation for informatics enthusiasts and everyone who are interested in 
informatics education for schools to think about the proposed Bebras idea, to investigate 
the local situation and to join the initiative. 
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Fig. 3. Cloud of papers’ keywords. 
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3.1. The Bebras Contest is Effective Enough in Promoting Informatics 
Concepts and Computational Thinking in Children and Young People 


One of the most important objectives of the Bebras challenge is to support deeper 
understanding of informatics rather than reproduction of previously learned facts. In 
the Bebras tasks, we can involve concepts of informatics like algorithms and pro¬ 
grams: sequential and concurrent; data structures like heaps, stacks and queues; mod¬ 
elling of states, control flow and data flow; human-computer interaction and graphics 
(Futschek and Dagiene, 2009). Creative and interesting tasks are the main driver for 
the Bebras contests. Understanding and handling the basics and foundations of in¬ 
formatics is more important than knowing a lot of technical details. This contest is 
informal learning-by-doing while solving attractive and challenging tasks (Haberman 
et al., 2011) 

Relevant aspects of informatics concepts to all schools can be part of the challenge 
problems. Especially for younger contestants, it is necessary to formulate the tasks in 
an adequate way. A contest may introduce a variety of even advanced concepts in a very 
short time. The main focus while preparing a contest should be given to the development 
of good tasks that also can be used by the students and teachers in their further learn¬ 
ing and teaching activities (Dagiene and Futschek, 2010). Teachers should learn how 
to explain what is behind one or another Bebras task. We found that the Bebras tasks 
are well-balanced according to the cognitive skills’ domains (Bloom taxonomy): at the 
most tasks are in the high categories Understanding, Applying, Analysing and Evaluat¬ 
ing (Dagiene and Stupuriene, 2014). 

The common practice for the contest is to include a larger proportion of tasks related 
to algorithms and programming concepts. The average ability to solve a certain task is 
largely independent of the school system. Students seem to struggle with abstract thin¬ 
king (Dagiene et al, 2014a). 

Initially, the Bebras contest was dedicated to introduce both informatics and infor¬ 
mation technology. Later, starting from 2011, the Bebras contest became a complex 
approach for deeper learning informatics and developing a computational thinking. The 
research area was expanded and more articles were appearing in the informatics educa¬ 
tion research community. 

In the context of primary school, Bebras tasks can be used with a focus on program¬ 
ming (Gujberova and Kalas, 2013). It is needed to understand what cognitive demands 
pose these tasks to primary school pupils, what the pupils consider difficult and what 
they consider easy. These questions could help all of us in searching the better peda¬ 
gogy for the primary programming education. It is important to emphasis on developing 
computational thinking, programming, designing computational systems and other basic 
concepts of informatics. 

To master complex education scenarios, e.g. learning algorithms, data structures, 
programming or data modelling, the educational technique of “didactic reduction” is 
very useful. For getting better educational results at the end, the teacher may reduce 
details in the learning contents at the beginning. The goal of learning computational 
thinking is to understand principles and concepts in informatics. Some Bebras tasks are 
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used as examples of learning scenarios with different forms of extreme didactic reduc¬ 
tion (Futschek, 2013). 

The promoting idea of the Bebras contest is not to test students’ knowledge, but 
in long time period, it looks like that this contest can be used for assessment. Authors 
investigated the “task based assessment” approach of the international Bebras contest 
by classifying (according to computational thinking categories) the conceptual content 
and question structure of the Bebras tasks during the years 2010-2014 (Barendsen el al, 
2015). In the Bebras tasks, algorithms and data representation were found to be the 
main concept categories (for the K-9 education). There was constructed a classifica¬ 
tion for question types (only for algorithms categories). Among the algorithms-related 
tasks, verification questions were the most common in all years except 2014. It makes 
some sense that verification questions are popular for a contest structured around multi¬ 
ple-choice questions. The predominant presence of algorithmic aspects in Bebras tasks 
might make this “task based assessment” interesting for K-9 teachers. 

Computational thinking is presented as one of the main strands of CSTA K-12 Stan¬ 
dards (Seehorn, 2011). One of the most important required cognitive operation, (men¬ 
tioned in this CSTA document) is: use visual representations ofproblem states, structures, 
and data. This might inspire the Bebras community to develop tests for computational 
thinking based on the Bebras questions. For this purpose, after a thorough analysis of the 
questions of all contests up to now, some suitable sets could be assembled to form test 
booklets that indeed could measure the computational thinking standards of the CSTA 
(Hubwieser and Miihling, 2015). 

Bebras tasks could be used to a wider community than it is planned, not only for 
school students. Research done by Dolgopolovas et al. (2015) aimed to evaluate novice 
college students in Software Engineering. The first step of the research process was 
developed as a homogenous dichotomous test using the Bebras contest tasks as the test 
questions. By solving the presented tasks, students needed to employ their computation¬ 
al thinking skills, which the research team intended to measure as a latent trait. Ten com¬ 
putational thinking tasks were selected and the average results of the solving these tasks 
were 54.2%. The next steps of the test process included questions requiring students to 
identify informatics concepts which were presented in the first step test tasks. The type 
of activity of the students during this study (game-like informatics task with the concepts 
“encoded” into it) was not only to test students but may be useful as a learning activity 
for informatics/engineering students. The activity itself may be highly motivational to 
learn new informatics concepts. 

In psychometrics, item response theory (IRT) is a paradigm for the design, analy¬ 
sis, and scoring of tests, questionnaires, and similar instruments measuring abilities, 
attitudes, or other variables. Sets of the Bebras tasks represent a psychometric test that 
measures certain joint psychometric constructs (and competencies). While the results are 
promising for the Bebras challenge, the main goal was to propose a methodology in the 
form of a specific process of evaluation and a proof of concept for this. The results show 
that this methodology works and might result one day in a set of empirically founded 
competencies in the field of CT (Hubwieser and Miihling, 2014a). Similarly, to PISA, 
the tasks are proposed by the members of an international board of experts. First, it is 


34 


V. Dagiene, G. Stupuriene 


necessary to stress that the Bebras contest is not a PISA like study. This might be the 
most probable reason for the low number of question combinations that are measure 
some homogenous psychometric constructs in a suitable way. Additionally, the Bebras 
contest does (at least officially) not require any pre-requisite knowledge (Hubwieser and 
Muhling, 2014b; Hubwieser and Miihling, 2015). 


3.2. How to Develop the Quality of the Bebras Contest? 


There are many quality aspects in the Bebras challenge: first, tasks (developing crite¬ 
ria for good tasks, visualisation, correctness, etc.), second, online contest management 
systems (effective, user friendly, flexible), third, importance of learning (informatics 
concepts), etc. 

Task categorisation and classification is one of the early research questions. The pro¬ 
posed topics are of the following major groups (Opmanis et al., 2006): 

1) General logic. 

2) ICT in everyday life. 

3) Practical and technical issues. 

4) Information comprehension. 

5) Algorithms and programming. 

6) Mathematics underlying informatics. 

7) History and trivia. 

Later (in 2008) the Bebras tasks’ categories were revised, and a modified classifica¬ 
tion came out (Dagiene and Futschek, 2008): 

1) Information comprehension. 

2) Algorithmic thinking. 

3) Using computer systems. 

4) Structures, patterns and arrangements. 

5) Puzzles. 

6) Social, ethical, cultural, international, and legal issues. 

This classification is used until now, though it has a lot of critics and limitations that 
are more and more visible. This category system has shown to be too coarse to be useful 
for task set assessment. In order to develop a fine-grained classification, a more complex 
system is needed to combine the following aspects (Pohl and Westmeyer, 2015): 

(1) The computational thinking vocabulary lists. 

(2) The fundamental ideas of informatics are based on three “master ideas”: algorith- 
misation, structured dissection, and language (later extended to formalisation). 

(3) Areas of “content competences” used to define educational standards for infor¬ 
matics education in middle school: information and data; algorithms; languages 
and automata; informatics systems; and informatics, man, and society. 

Kalas and Tomcsanyiova proposed different categorisation of the Bebras tasks in 
2009. They developed four components of informatics education and suggested to use 
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them for categorising tasks: digital literacy; programming; problem solving; and data 
handling (Kalas and Tomcsanyiova, 2009). 

Contests can be a form of learning and a way to test children’s (age 8-9) knowledge 
(Tomcsanyiova and Tomcsanyi, 2011). Children have digital skills, but it is important 
to prepare the tasks for them in very special way. Pupils of these ages read more slowly 
and do not always understand the text correctly; children at this age need to work with 
concrete objects within the software and they do not understand abstraction; children at 
this age cannot focus on a task for a long time. 

There is coherence between the assignment of tasks and the errors students make 
(Hansky et al., 2011). A general classification scheme of typical error types developed 
by Weimer (psychology) was the basis of the study. Since the proposed categories did 
not cover all errors that were identified during the analysis, the authors suggested to 
extend the categorisation with the error types mistake, error of false conclusion and 
strategy error. Based on the commonalities of tasks, it can be concluded that mistakes 
occurred if the task setting was closely related to the world of experience, and if it 
activated the cognitive process level memory. The introducing information should be 
provided more than once, e. g., as text plus diagram. Errors of false conclusion ap¬ 
peared together with the requirement of conceptual knowledge. The less the students 
know about concepts, the more likely are these errors. Therefore, there should be more 
training in the area of concepts. 

It is equally important to pay attention to formulation (text phrases) of the incor¬ 
rect answers in multiple-choice tasks as their quality may considerably influence task 
difficulty - this was the main result and message of the research paper published by 
Vanicek (2014). 

Tasks are the keystones of the Bebras challenge. Some researchers have focused 
on phenomena that are directly related to informatics concepts (e.g., algorithms, data 
structures). One of the interesting objectives was to categorise the pictures of Bebras 
tasks according to their function, content and type. When categorising pictures accord¬ 
ing to their content, Tomcsanyiova and Kabatova (2013) proposed three categories 
(problem setup pictures, pictures depicting an informatics concept, and pictures show¬ 
ing a user application interface) covering the majority of task pictures. Understand¬ 
ing and interpreting diagrams are at the core of solving process, whatever the answer 
requires a direct manipulation or mental process. In this sense, the Bebras tasks are 
“situational images”. 

Many of the Bebras tasks have diagrams or schemas to visualise the content of a 
question. A majority of diagrams are schematic and give an abstract representation of 
the problem to be solved. Most of those schematic diagrams are conventional represen¬ 
tations used in informatics: graph, tree, grid and also diagrams showing processes or 
structural relationship between objects. Another frequent type of diagrams is icon and 
drawing. Icons represent objects and highlight on some of their properties by using a 
visual code for properties values (Tort and Drot-Delange, 2015). It is important to enable 
blind pupils to participate in the Bebras contest as well. The results achieved by blind 
pupils and factors affecting these participants as well as suggested rules of adaptations 
were analysed in one of surveys (Jaskova and Kovacova, 2015). 


36 


V. Dagiene. G. Stupuriene 


In order to improve the quality of tasks, developers should use short sentences; 
words or phrases repeatedly; clear definitions; a one-to-one relationship between words 
and objects; appropriate analogies; and unambiguous wording (Pohl and Hein, 2015). 
Interactivity of tasks makes the contest more attractive if interactivity means manipula¬ 
tion with mouse. If tasks stem out from real life situations, it will be appreciated by older 
contestants. Younger will enjoy motivation through the fairy tale character of Beaver 
(Vanicek, 2014). Another aspect is guessing - students merely choosing an answer at 
random instead of leaving it blank. A guess is frequent in multiple-choice questions: 
about 10% of contestants try to guess the right answer and boys do it more frequently 
(Dagiene et al., 2014a; Dagiene et al., 2015). 

Pupils are more disposed to solve tasks (or guess the answer) than to make “no 
answer” choice. The simplest (tasks of low difficulty and complexity) are those called 
in competition slang “consolation tasks”. Their solution is possible for practically each 
participant. The existence of such tasks in competition gives a positive emotion even to 
those who solved few tasks only. Researchers have noticed that the number of such tasks 
proposed for competition was negligible (Yagunova et al., 2015). 

Case study from Turkey (Kalelioglu et al, 2015) was based on school student perfor¬ 
mance. Students were successful with an average score of 65.01 (scores varied between 
0 and 135). Results can be regarded as successful because of the quality and level of dif¬ 
ficulty of the questions, as well as the requirements for algorithmic skills to solve them. 

Other case study from Czech Republic and Slovakia (Tomcsanyi and Vanicek, 2009) 
introduces the contest and compares the situation from the point of view of motivation, 
organisation, publicity, realisation, and number and structure of participating students in 
these two countries. 

Statistical data from Germany (Schlueter and Brinda, 2009) was used in the study 
directed to modelling the competence requirements of tasks in informatics. Initially, the 
focus was on the relevance of a catalogue of criteria affecting the task difficulty where 
relevance is supposed to mean the practicability as well as the interrelation between 
criteria and competence requirements. 

Since pretesting is impossible, it is useful to find out what kinds of tasks were mis¬ 
placed and develop some guidelines for predicting the difficulty level. Vegt (2013) pro¬ 
posed the use of a questionnaire that can be used to discover probable causes of diffi¬ 
culty. The answers can be used to compare questions and to discuss the estimation of the 
difficulty level of the question. 

The same problem with difficulty level is mentioned in research where authors found 
that a priori evaluation of problems by the organisers does not correspond to the diffi¬ 
culty of the task experienced by the participants (Yagunova et al., 2015). The researchers 
underestimated the complexity for younger pupils and overestimated the difficulty for 
older. The pupils, especially primary school kids, frequently underestimate the complex¬ 
ity of the tasks. Researchers suppose that the task length was the factor which determined 
a large number of refusals in junior classes, even though the tasks were not complex, in 
fact. Another complicating factor is interface peculiarities of a competition. Tasks con¬ 
taining long texts may have not enough space to be presented on one screen. For senior 
students, the length of text is not an extra complicating factor. Having made mistake 
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with assessment of task difficulty for junior schoolchildren, the experts evaluated best of 
all the complexity of tasks for them. That is to say, the experts do not evaluate accurately 
the difficulty of tasks for juniors and their complexity for seniors. 

Bloom’s taxonomy can help to determine the difficulty level of tasks (Dagiene and 
Stupuriene, 2014). There is a slight mismatch between the difficulty level of the tasks 
used in the contest and students’ actual abilities. To some extent the actual difficulty 
level seems to correlate with the classification in Bloom’s taxonomy. It might hence be 
worthwhile to classify each task picked for a given age group before assigning the diffi¬ 
culty level (Dagiene et al., 2014b). In other point, it is necessary to check whether or not 
the difficulty perceived by the pupils was substantially different from that estimated by 
those who proposed the tasks. This is important as a feedback in order to gain knowledge 
to be used both in ranking participants and in organising future editions of the contest. 
For this purpose, IRT was used (Bellettini et al., 2015). 

Contest management systems (CMS) are used for running the Bebras contests. Many 
countries had developed their own CMSs and discussing their functionality and other 
features. Currently, there are over ten of Bebras CMSs. Quality issues of CMSs are very 
important research questions within Bebras community. However, very little evidence is 
published at the moment. A Slovenian research team offered a model of a new manage¬ 
ment system which has to be: 

(1) High performance. 

(2) Scalable. 

(3) Fault-tolerant. 

The system was implemented and localised to three languages (English, Slovene and 
Serbian) (Kristan et al., 2014). 


3.3. The Bebras Contest for Formal and/or Informal 
Informatics Education in Countries 


Striving to fulfil ambitious goals in developing modern informatics education, policy mak¬ 
ers suggest various strategies. One of them is organising a range of gameful contests for all 
student age groups (Kalas and Tomcsanyiova, 2009; Kalas, 2014). Reviewing the develop¬ 
ment of Europe informatics education, the Bebras challenge is seen as one of informal ways 
of introducing pupils to informatics topics. Informatics, the science of algorithmic process¬ 
ing, representation, storage and transmission of information, is an important discipline in 
the knowledge society and should be introduced to secondary or even to primary schools. 

The challenge should help children to get interested in informatics and to stimulate 
thinking about contributions of informatics to science at the very beginning of school 
(Dagiene, 2010). The Bebras contest may play an important role in creation of school 
curricula from the “bottom”, from basic elements and individual questions upon which 
broader informatics concepts may be illuminated (Vanicek, 2013a and 2014). 

Many competitions in computing and information technology are mainly intended 
for talented students and focus on practical areas such as developing algorithms and 
programs. The Bebras challenge has instead provoked students to solve problems from 
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a broad range of informatics topics without any programming skills required. Usually 
many pupils enjoy friendly organised competitions, and the Bebras challenge can be 
used to attract children to the informatics domain covered by the contest tasks (Bell 
etal., 2011; Bilbao etal., 2014; Heintz etal., 2015; Maytarattanakhon etal., 2015; Syslo 
and Kwiatkowska, 2015). In another paper, Galik and Kalas (2009) have stated that the 
Bebras is a contest focused on promoting interest in digital technologies and informatics 
for all students. Moreover, following many other researchers, the Bebras contest encour¬ 
ages students to use modern technologies in their learning activities more intensively 
and creatively (Syslo, 2011; Lonati et al., 2011). 

According to French case, the Bebras contest is a great tool for working and thinking 
about informatics education (Tort and Drot-Delange, 2013; Baron et al., 2014). First, it 
is a way to raise questions among educational actors about informatics education. Teach¬ 
ers who engage their students in the contest come from other subject areas. When they 
spend one hour on the contest, they give a place to Informatics concepts and to compu¬ 
tational thinking in their own teaching. 

Informatics is still a male-dominated discipline, but results suggest that girls aged 
10-13 manage equally well (or even better) than boys in this contest (Dagiene et al., 
2014a). In the lower secondary school age, there are no significant differences between 
boys and girls in their interests and performances (Kalas and Tomcsanyiova, 2009). 

The results of several researchers indicate that both girls and boys can have a suc¬ 
cessful and rewarding experience exposed while solving the Bebras tasks (Dagiene et al., 
2015). Boys and girls show different results in tasks which require spatial thinking, it 
takes the same amount of time to solve a task but boys do it better. Pupils’ performance 
of tasks increases with the age but boys of lower grades have almost equal results as 
girls of upper grades. However, the case study of Germany shows that the boys showed 
a significant better performance compared to the girls. Also, the pairs performed better 
than the singles (Hubwieser and Muhling, 2015). 

A long-standing problem is discussed by several researchers: how to teach an intro¬ 
duction to theoretical informatics to secondary school pre-service and in-service teach¬ 
ers of informatics who lack sufficient mathematical background. An attractive method 
of solving five tasks from the Bebras contest was suggested to teachers in one research 
project (Bezakova and Winczer, 2011). Teachers were asked to play the task, and through 
that play they were guided to discover informatics concepts. Each new informatics con¬ 
cept was connected to real situations of a particular task. 

Bebras tasks can be adopted or modified to many other activities, for example, to 
measure a difference in programming abilities (Ruf et al., 2014), to stimulate students’ 
online performance (Kabatova et al., 2013). 


4. A Model for Concept Based Learning 

The “concept” can be described as extensive piece of information on particular object 
existing in human sense. The content of a concept can vary a lot as it depends on person¬ 
al experience. Concepts of informatics are strongly related to our intention what basics 
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we would like to develop to children and at which age. In formal sciences, “concept” 
is defined as an abstract idea which generalises separate objects, defines attributes and 
some relations between objects. 

Informatics concepts play a central role for understanding fundamentals of com¬ 
puters, information technology, software, hardware, and information systems. Concepts 
that should be introduced in general school education and represented in assignments 
and tasks, are still a subject of research and discussions (Hromkovic, 2006; Kalas and 
Tomcsanyiova, 2009; Vanicek, 2013b). It is problematic to decide what we should in¬ 
clude in informatics education for primary and secondary schools. Some reasons for that 
could be as follows: 

(1) Informatics is relatively young and rapidly evolving science. 

(2) There is a variety of different practical applications of informatics and that over¬ 
runs core theoretical and scientific concepts. 

(3) There is no common agreement (framework) on what should be introduced in 
school from the theory of informatics, or whether it should be introduced at all. 

It is important to bring informatics concepts to pupils in a very attractive way. Bebras 
challenge is one of the possibilities to bring informatics education in an informal way 
(Armoni and Gal-Ezer, 2014; Dagiene, 2011). The Bebras tasks are a major source in 
introducing informatics concepts. According to common agreed definition, each Bebras 
task should contain at least one informatics concept. The tasks have to concentrate on 
smaller learning items. Due to the independence from specific systems the focus of the 
tasks is not on working in real systems but on understanding of the principles, ideas and 
concepts that are involved. 

Informatics concepts can be learned by a spiral way. The spiral cycle begins with 
a certain informatics concept, which is the key idea what we want to teach the pupils 
(Fig. 4). By adding a funny and interesting story, we create a task. By using gamification 
(application of game principles in non-game contexts), and by adding interactive com¬ 
ponents (dragging, dropping, etc.) we can develop a task for the Bebras challenge. 



Fig. 4. From an informatics concept to a pupil understanding of it (from a teacher point of view). 
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The next big step is to solve a task (this is pupils’ matter), however due to be of short 
time a pupil’s main attention is paid to get the correct answer. The learning and under¬ 
standing process of informatics concepts will come later, actually after practice to solve 
many of these tasks. A teacher’s role is very important for strengthening the understand¬ 
ing of informatics concepts. Teachers can help pupils to clarify task solutions, to explain 
why it is informatics, to provide resources for readings and discussions. 

Learning concepts at an early age are very important for deeper understanding of 
various informatics topics. The Bebras task focus on informatics concepts supports un¬ 
derstanding of informatics phenomena and developing computational thinking. Devel¬ 
opment of Bebras tasks is actually based on operational definition of computational 
thinking which suggests that computational thinking is a problem-solving process that 
includes (but is not limited to) the following characteristics (ISTE&CSTA, 2011): 

• Formulating problems in a way that enables us to use a computer and other tools 
to help to solve them. 

• Logically organising and analysing data. 

• Representing data through abstractions such as models and simulations. 

• Automating solutions through algorithmic thinking (a series of ordered steps). 

• Identifying, analysing, and implementing possible solutions with the goal of 
achieving the most efficient and effective combination of steps and resources. 

• Generalising and transferring this problem solving process to a wide variety of 
problems. 

The computer science teacher association in USA (CSTA) goes on by adding broader 
attitudes like ability to deal with complexity and open ended problems, tolerance for 
ambiguity, and ability to work with others to achieve a common goal. 

Summarising, the operationalisation of computational thinking and informatics prob¬ 
lem solving seems closely interwoven with ways to deeper learning linked to modern 
digital technologies and practices. 


Conclusions 

Analysis of the 56 research papers on the Bebras challenge of informatics and computa¬ 
tional thinking has discovered several evidences about the impact of the Bebras contest 
on the national curricula, especially on informatics education implementation in formal/ 
informal levels. Learning through solving small pieces of concept-rich tasks, or a flipped 
learning, fits better to the digital era society. 

The quality of the Bebras challenge is increasing by expanding activities for sup¬ 
porting informatics education and developing more powerful tasks. Actually goal is to 
develop rich tasks to have at their core, the opportunity to develop an understanding of, 
explore and employ informatics concepts. Several research groups have continued their 
work in this direction. 

There is evidence about Bebras challenge as an effective tool in promoting problem 
solving skills and computational thinking. For the Bebras model, a connection to compu- 
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tational thinking is quite a new challenge and needs more efforts in preparation strategy 
for dealing with it. Thus, a good starting point could be the different points of view that 
the different professional groups have: pedagogic (teachers), pragmatic (designers of the 
challenge and tasks) or technical (implementers of tools). 

Following the literature analysis and observing the Bebras contest implementation 
for more than ten years, a community building Bebras model was elaborated. The model 
bridges both international and national implementation levels and consists of various 
components. 

Bebras tasks are cornerstones of the challenge. A lot of research questions can be 
raised in concern to developing the good Bebras tasks, their difficulty levels, validation, 
motivation pupils for learning, supporting informatics understanding, developing new 
tools for interactive tasks, etc. 

This paper proposed an original development of informal informatics education 
model through questioning and solving tasks. Embracing concept-based learning means 
that we need to consider new design metaphors for future learning. However, we must 
remember that students (and teachers) will still need motivation to learn things. There¬ 
fore, this paper argues for greater motivation reached by attractive and interactive short 
tasks for learning informatics concepts. Future work will be related on students’ long¬ 
term participating in the Bebras challenge in order to show evidence of the power of 
concept-based learning of informatics and computational thinking. 
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